Towards Transparent and Grounded Visual AI Systems

My research goal is to build transparent and grounded AI systems. More specifically, my research tries to answer the question -- Do deep visual models make their decisions for the "right reasons"? In my dissertation, I try to answer this question in two ways: 1. Visual grounding. Grounding is essential to build reliable and generalizable systems that are not driven by dataset biases. In the context of the task of Visual Question Answering (VQA), we would expect models to be visually grounded, i.e., looking at the right regions in the image while answering a question. I address this issue of visual grounding in VQA by proposing a) two new benchmarking datasets to test visual grounding, and b) a new VQA model that is visually grounded by design. 2. Transparency. Transparency in AI systems can help system designers find their failure modes and provide guidance to teach humans. I developed techniques for generating explanations from deep models that give us insights into what they are basing their decisions on. Specifically, I study the following -- a) what parts of the inputs VQA models focus on while making a prediction, b) a new counter-example explanation modality where a VQA model has to identify images for which a given question-answer is not true, c) counterfactual visual explanations and how we can use such explanations to teach humans, and d) causal concept explanations (explaining “zebra” class prediction in terms of human-understandable concept “stripes”) by reasoning about the causal relationship between concept explanations, images and classifier predictions.Ph.D

Prediction of strength enhancement of subgrade soil reinforced with geotextile using artificial neural network and M5P model tree

Geosynthetics layers are being implemented as reinforcement to enhance the strength of subgrade soil (which is calculated in terms of CBR). Present research work, aims at investigating the strength enhancement in terms of CBR through experimental study. Experiments were conducted on subgrade soil reinforcing it with single and double layer woven and non-woven geotextile layer were placed at depth M/3, M/2 and 2/3M from the top of CBR specimen, where Mis height of CBR specimen. Result indicate that woven geotextile offers more strength to subgrade soil than non-woven geotextile, further as depth of placement of reinforcement increases from top lesser is increase in strength for both the geotextile. Strength also increases when double layer was placed in comparison to single layer for both the geotextile. ANN and M5P was used to predict the CBR value, result suggest improved performance of ANN over M5P for present data

FastDocFastDoc: Domain-Specific Fast Pre-training Technique using Document-Level Metadata and Taxonomy

As the demand for sophisticated Natural Language Processing (NLP) models continues to grow, so does the need for efficient pre-training techniques. Current NLP models undergo resource-intensive pre-training. In response, we introduce $FastDoc$ (Fast Pre-training Technique using Document-Level Metadata and Taxonomy), a novel approach designed to significantly reduce computational demands. $FastDoc$ leverages document metadata and domain-specific taxonomy as supervision signals. It involves continual pre-training of an open-domain transformer encoder using sentence-level embeddings, followed by fine-tuning using token-level embeddings. We evaluate $FastDoc$ on six tasks across nine datasets spanning three distinct domains. Remarkably, $FastDoc$ achieves remarkable compute reductions of approximately 1,000x, 4,500x, 500x compared to competitive approaches in Customer Support, Scientific, and Legal domains, respectively. Importantly, these efficiency gains do not compromise performance relative to competitive baselines. Furthermore, reduced pre-training data mitigates catastrophic forgetting, ensuring consistent performance in open-domain scenarios. $FastDoc$ offers a promising solution for resource-efficient pre-training, with potential applications spanning various domains.Comment: 38 pages, 7 figure

Assessment of efficacy and safety of artesunate plus sulfadoxine pyrimethamine combination for treatment of uncomplicated falciparum malaria

Background: Resistance of Plasmodium falciparum to antimalarial drugs is common in India. World Health Organization (WHO) recommends artemisinin‑based combination therapy (ACT) to counter the development of resistance in P. falciparum. WHO recommends that ideally antimalarial drug treatment policy or guidelines should be reviewed regularly and updated at least once every 24 months. In consideration to the above recommendation, we planned to conduct the following study. The objective was to determine the efficacy and safety of artesunate + sulphadoxine‑pyrimethamine (AS + SP) in patients with uncomplicated P. falciparum malaria.Methods: The study included 60 patients of uncomplicated P. falciparum. Each patient received AS + SP as per WHO guidelines. Diagnosis was confirmed by peripheral blood film. All patients were followed‑up on days 1, 3, 14, and 28 for detailed clinical and parasitological examination.Results: Of a total 60 patients, 55 patients were followed‑up for 28 days. Remaining 5 patients were lost in follow‑up. As per protocol analysis, 91% (50) of patients had demonstrated adequate clinical and parasitological response. Remaining 9% (5) had treatment failure in which 5.5% (3) had late parasitological failure and 3.6% (2) had late clinical failure. In our study, mean parasite clearance time was 45.2 ± 4.2 hrs.Conclusion: AS + SP is safe and effective drug for the treatment of uncomplicated falciparum malaria. However, the efficacy of this ACT needs to be carefully monitored periodically since treatment failure can occur due to resistance

Effect of gabapentin on haloperidol induced inhibition of conditioned avoidance response in rat

Background: Haloperidol, an antipsychotic adversely affects acquisition and retention of a learned task. We decided to test the effect of Gabapentin, a new anti-epileptic drug using conditioned avoidance response model with cook’s pole climbing apparatus and haloperidol.Methods: Four groups of six rats were taken for this purpose. All the rats were first given drugs for five days and then trained for a period of 15 days. Gabapentin was given in a dose of 100mg/kg intra peritoneal, while haloperidol was given 0.5mg/kg intra peritoneal.Results: At the end of the training duration rats in the vehicle and gabapentin treated group achieved ≥85% acquisition responses. While the haloperidol and haloperidol + gabapentin group did not achieve the desired percentage of learning. A learning curve was plotted by using the percentage of conditioned responses in each group versus number of days. The mean ± SD percentage of conditioned responses of day 14 and 15 were for haloperidol group 26.19 ±11.90, for vehicle group 86.90 ± 4.29, for the gabapentin treated group 95.24 ± 2.38 and for the gabapentin + haloperidol group 46.42 ± 12.20. These figures and the learning curve suggest that gabapentin treated rats had a better acquisition response and haloperidol depressed learning.Conclusions: At the end of study duration we found that gabapentin significantly improved the acquisition response than the vehicle control group. Also haloperidol depressed the acquisition response. Gabapentin did not lead to reversal of haloperidol induced depression of acquisition process